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ABSTRACT 

Objectives: Examine the probability of new active 
substances (NASs) approved in Canada between 1 
January 1997 and 31 March 2012 acquiring a serious 
postmarket safety warning. 
Design: Cohort study. 

Data sources: Annual reports of the Therapeutic 
Products Directorate and the Biologic and Genetic 
Therapies Directorate; evaluations of therapeutic 
innovation from the Patented Medicine Prices Review 
Board and Prescrire International; MedEffect Canada 
website. 

Interventions: Postmarket regulatory safety warning or 
withdrawal from market due to safety reasons. 
Primary and secondary outcome measures: 

Compare the probability of acquiring a postmarket safety 
warning in Canada in four different groups of drugs: (1) 
traditional medications versus biologies; (2) medications 
that offer significant new therapeutic benefits versus 
those that do not. Determine how well the type of review 
that an NAS received from Health Canada predicted the 
product's postmarket therapeutic value. 
Results: The probability of a traditional NAS acquiring a 
serious safety warning and/or being withdrawn was 
29.9% (95% CI 21.8% to 40.2%) vs 27.3% (95% CI 
18.2% to 39.7%) for an NAS of biological origin 
(p=0.47, log-rank test). For medications that were 
significant therapeutic advances the probability was 
40.2% (95% CI 24.5% to 60.9%) vs 33.9% (95% CI 
26.4% to 42.7%) for those that were not (p=0.18, 
log-rank test). Health Canada was 77.4% accurate in 
predicting the therapeutic importance of an NAS. 
Conclusions: There was no difference in postmarket 
regulatory safety action between traditional medications 
and biologies and no difference between drugs with 
significant therapeutic benefits and those without. 
Although these results draw on Canadian data, they are 
likely to be relevant internationally. Further research 
should assess whether the current level of premarket 
safety evaluation is acceptable. 
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INTRODUCTION 

Drug safety is becoming a topic of increasing 
concern in Canada. In July 2008, the federal 
government officially launched the Drug 



Systematic study of the postmarket regulatory 
safety warnings comparing groups of drugs: bio- 
logics versus traditional medicines and drugs 
with significant therapeutic advances versus 
drugs without significant therapeutic advances. 
Comparison of premarket regulatory evaluation 
of therapeutic advance with postmarket 
evaluation. 

Unclear what criteria Health Canada uses to 
decide to issue serious safety warnings. 
Unknown date on which a drug actually mar- 
keted as opposed to date approved. 
Postmarket therapeutic evaluation of all new 
drugs could not be determined. 
Presence of a postmarket safety warning does 
not necessarily equate with the overall safety of a 
drug. 



Safety and Effectiveness Network (DSEN) 
designed to connect researchers throughout 
Canada in a virtual network to conduct post- 
market drug research 1 and stimulate 
research to study the impact of drug use in 
the real-world setting. 2 In 2010, the Health 
Council of Canada released a discussion 
paper that drew on international best prac- 
tices for recommendations about how 
Canada could improve its developing system 
of active pharmacosurveillance. 3 In 2011, the 
Auditor General reported that Health 
Canada is slow to assess potential safety issues 
and can take more than 2 years to provide 
Canadians with new safety information. 4 
Most recently, legislation has been intro- 
duced that would give Health Canada the 
power to order additional safety testing for 
drugs already on the market. 5 

The increased focus on drug safety comes 
from a number of directions. In the USA, 
adverse drug reactions (ADRs) are estimated 
to result in between 76 000 and 137 000 
deaths/year, making ADRs the fourth to 
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sixth leading annual cause of death. 6 The Institute for 
Safe Medication Practices puts the number of annual 
deaths in the USA due to ADRs at 128 000 based on 
reports to the Food and Drug Administration (FDA). 7 
Since the Lazarou et al estimate was made in the late 
1990s, the reported serious adverse drug events 
increased 2.6-fold from 1998 to 2005 and fatal adverse 
drug events increased 2.7-fold during the same period 
while the total number of outpatient prescriptions went 
up by only 40%. 8 While there are relatively few drugs 
withdrawn from the market for safety reasons, 9 large 
numbers of people have been exposed to some of these 
products. In 2003, the year before rofecoxib (Vioxx) was 
removed from the market, it was the 10th most fre- 
quently prescribed medication in Canada. 10 

A recent analysis of drug safety in Canada found that 
almost one in four new active substances (NASs) approved 
between 1995 and 2010 either had a serious safety warning 
or was removed from the market for safety reasons (An 
NAS is a molecule never previously marketed in any form 
in Canada. This designation is given to all molecules 
meeting the definition and therefore should not be seen 
as creating a division between 'new' and 'old' drugs). This 
figure increased to more than one in three for products 
that received a priority review, that is, products that Health 
Canada felt might provide an effective treatment of a 
disease for which no drug is presently marketed or a sig- 
nificant increase in efficacy and/ or significant decrease in 
risk over existing therapies. 11 Priority reviews have a time- 
line of 180 days versus the standard timeline of 300 days. 12 

This study compares postmarket regulatory safety 
action in four groups of drugs: (1) traditional medica- 
tions (those derived from chemical manufacturing) and 
biologies; (2) medications that offer significant new 
therapeutic benefits and those that do not. Biologies are 
large molecules synthesised from living organisms and 
typically administered intravenously. As such, they may 
have a significantly different safety profile compared to 
traditional small molecule medications that come from 
chemical synthesis and are usually ingested orally. 
Regulators may be willing to approve drugs that offer sig- 
nificant therapeutic advances with more uncertainty 
about their safety compared to drugs that are not a sig- 
nificant therapeutic advance. 

Specifically, this study attempts to reject the following 
two null hypotheses: (1) there is no difference in the 
postmarket safety profile of traditional versus biological 
medications, (2) there is no difference in the postmar- 
ket safety profile of drugs with significant therapeutic 
advances versus those without. A secondary objective was 
to determine how well the type of review that an NAS 
received from Health Canada predicted the product's 
postmarket therapeutic value. 

METHODS 

A list of NAS approved between 1 January 1997 and 31 
March 2012 was compiled from the annual reports of 



the Therapeutic Products Directorate (TPD) and the 
Biologic and Genetic Therapies Directorate (BGTD; 
henceforth collectively referred to as the TPD) — avail- 
able by directly contacting the directorates at 
publications@hc-sc.gc.ca. For each product, the follow- 
ing information was abstracted: generic name, brand 
name, manufacturer, indication, date of notice of com- 
pliance (NOC — marketing authorisation), type of review 
(priority or standard) and type of product (traditional 
or biological). The TPD annual reports only gave the 
type of product (traditional or biological) from 2000 
onwards. 

Two sources were used to determine the postmarket 
therapeutic value of the NAS: the annual reports of the 
Patented Medicine Prices Review Board (PMPRB) avail- 
able online from 2003 to 2011 at http://www.pmprb- 
cepmb.gc.ca/english/View.asp?x=91 and for previous 
years by directly contacting the PMPRB at pmprb@ 
pmprb-cepmb.gc.ca and the online reviews published by 
Prescrire International up to 14 February 2013 http:// 
english.prescrire.org/en/. These sources were chosen 
because their evaluations are unambiguous and there- 
fore do not require any subjective interpretation and 
they are both available in English. 

The PMPRB is a federal agency that is responsible for 
calculating the maximum introductory price for all new 
patented medications introduced into the Canadian 
market. As part of the process of determining the price, 
its Human Drug Advisory Panel determines the thera- 
peutic value of each product it reviews. Up until the end 
of 2009, NASs were classified into two groups: (1) break- 
throughs or substantial improvement and (2) moderate, 
little or no therapeutic improvement. Since 2010, NASs 
are classified as breakthrough or substantial improve- 
ment, moderate improvement (primary or secondary) 
and slight or no improvement. For the purpose of this 
study, products that were deemed breakthrough and sub- 
stantial improvement were termed 'significant thera- 
peutic advance' and products in other groups were 
termed 'no therapeutic advance'. The change in the 
PMPRB system starting in 2010 was meant to provide a 
finer gradation in the 'moderate, little or no therapeutic 
improvement' group and as such did not affect the 
dichotomous classification used here between 'signifi- 
cant therapeutic advance' and 'no therapeutic advance'. 
In some cases, the PMPRB annual reports indicated that 
the therapeutic value of the product was still being 
determined, and in those cases the PMPRB was con- 
tacted directly to determine the final classification. 

If the PMPRB had not considered a product, then its 
therapeutic value was determined from Prescrire evalua- 
tions. Prescrire rates products using the following cat- 
egories: bravo (major therapeutic innovation in an area 
where previously no treatment was available); a real 
advance (important therapeutic innovation but has lim- 
itations); offers an advantage (some value but does not 
fundamentally change the present therapeutic practice); 
possibly helpful (minimal additional value and should 
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not change prescribing habits except in rare circum- 
stances); nothing new (may be a new molecule but is 
superfluous because it does not add to clinical possibil- 
ities offered by previously available products); not 
acceptable (without evident benefit but with potential or 
real disadvantages); and judgement reserved (decision 
postponed until better data and more thorough evalu- 
ation). The first three Prescrire categories were defined 
as a significant therapeutic advance and the other 
Prescrire categories (except judgement reserved) were 
defined as no therapeutic advance. Previous work has 
shown a moderate level of agreement between the thera- 
peutic evaluations from the PMPRB and Prescrire. 13 

Safety warnings and drug withdrawals for the period 1 
January 1997 to 31 December 2012 were identified 
through advisories for health professionals on the 
MedEffect Canada website http://www.hc-sc.gc.ca/dhp- 
mps/medeff/advisories-avis/prof/index-eng.php. For 
each safety advisory or notice of withdrawal of a product, 
the date and reason were recorded. All serious safety 
advisories (those using bolded black print or boxed 
warnings) were included except for those dealing with 
the withdrawal of a specific batch or lot number due to 
manufacturing problems or those issued because of 
misuse of a drug (eg, an unapproved use) or medication 
errors (eg, a warning about remembering to remove a 
transdermal patch before applying a second one) . When 
necessary, notices on the MedEffect website were 
supplemented by searching on the product name in the 
Drug Product Database (DPD) http://webprod3.hc-sc. 
gc.ca/dpd-bdpp/index-eng.jsp. The DPD contains 
product-specific information on drugs approved for use 
in Canada as well as all products discontinued since 
1996. 

Troglitazone was approved but never marketed in 
Canada because of a dispute about its introductory 
price. There was no information about revocation of its 
NOC on the MedEffect website. The drug was removed 
from the US market in March 2000 and 15 March 2000 
was arbitrarily used as its withdrawal date in Canada. It 
was retained in the analysis because it was a product that 
was approved and then later shown to have side effects 
serious enough that it needed to be withdrawn. The 
TPD annual reports list infliximab as two separate NASs 
since it was approved for two different indications — 
Crohn's disease and rheumatoid arthritis — and there- 
fore it is included twice. 

The time between receipt of an NOC and a safety 
warning and/or withdrawal from the market was calcu- 
lated in days. If a drug received more than one serious 
safety warning, only the time to the first warning was 
used. Medians are reported for the time from NOC to 
serious safety warnings and/or withdrawal as these 
values are not normally distributed (Shapiro-Wilk test). 
Kaplan-Meier survival curves were calculated separately 
for the following comparisons: (1) biological versus trad- 
itional NASs and (2) NASs that were therapeutic 
advances versus those that were not. 



Health Canada gives a shorter priority review to drugs 
that it believes show evidence of providing a significant 
increase in efficacy or a significant decrease in side 
effects compared to other available agents for a serious, 
life threatening or severely debilitating illness or condi- 
tion, that is, drugs that Health Canada judges as provid- 
ing significant therapeutic gains. 12 Health Canada's 
accuracy in evaluating an NAS's therapeutic benefit was 
determined by comparing the review status given to the 
drug (priority vs standard) with the therapeutic evalu- 
ation from the PMPRB or Prescrire. 

There were no power calculations as the entire popula- 
tion of NAS was evaluated rather than just a sample. 
Calculations were carried out using Excel 2011 for 
Macintosh (Microsoft) and Prism V.6.0 (GraphPad 
Software). 

RESULTS 

Four hundred and six NASs were approved from 1 
January 1997 to 31 March 2012. Eighty-seven (21.4%) 
were subject to either a serious safety warning and/or 
were withdrawn for safety reasons: 72 (17.7%) had only 
serious safety warnings and 15 (3.7%) were withdrawn 
(8 had safety warnings first and 7 were withdrawn 
without any prior safety warning; online supplementary 
appendix lists all drugs with safety warnings and/or with- 
drawals). A notice that one product, gatifloxacin, had 
been withdrawn from the market never appeared on the 
MedEffect website and the withdrawal was only con- 
firmed on the DPD website. The median time to a first 
safety warning was 1094 days (IQR 551.8, 1812.5) and 
778 to withdrawal (IQR 486.5, 1119.5). 

Of the 298 NASs approved from 1 January 2000 to 31 
March 2012, 79 were biologies (60 with no safety warn- 
ings and 19 with safety warnings) and 219 were trad- 
itional medications (175 with no safety warnings and 44 
with safety warnings). The therapeutic status of 336 
NASs was determined from either the PMPRB (296 
NASs) or Prescrire (40 NASs) evaluations. Three 
hundred and five were not significant therapeutic 
advances (232 with no safety warnings and 73 with safety 
warnings) and 31 were therapeutic advances (20 with no 
safety warnings and 11 with safety warnings). Of the 70 
NASs where the therapeutic status could not be deter- 
mined, 66 had no serious safety warnings and 4 had a 
warning; none were withdrawn from the market. 

Twenty-two of the 31 NASs that were therapeutic 
advances received a priority review, whereas 67 of the 
305 NASs that were not significant advances also had a 
priority review. Overall, the review status was 77.4% 
accurate in determining the therapeutic rating of the 
NAS. 

The Kaplan-Meier curves show that the probability of 
a traditional NAS acquiring a serious safety warning 
and/or being withdrawn was 29.9% (95% CI 21.8% to 
40.2%) vs 27.3% (95% CI 18.2% to 39.7%) for an NAS 
of biological origin (figure 1, p=0.47, log-rank test). For 
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Figure 1 Kaplan-Meier estimate of new active substance 
survival: traditional medications versus biologies. 

medications that were significant therapeutic advances 
the probability was 40.2% (95% CI 24.5% to 60.9%) vs 
33.9% (95% CI 26.4% to 42.7%) for those that were not 
(figure 2, p=0.18, log-rank test). 

DISCUSSION 

The results of this study support the null hypotheses in 
both cases of no difference in safety warnings between 
traditional medications versus biologies and no differ- 
ence between drugs with significant therapeutic benefits 
versus those without. However, given the wide CIs, this 
conclusion should be regarded as tentative. Further 
research using a larger number of NASs might show stat- 
istically significant differences. Other comparisons 
between groups of drugs that have used safety warnings 
have similarly found no difference in postmarket safety. 14 
The finding that there was no difference in the prob- 
ability of acquiring a postmarket safety warning for NASs 
regardless of their level of therapeutic benefit is 
welcome news as it is one indication that more benefits 
are not being traded off against more harms. At the 
same time, it also calls into question the benefit : harm 
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Figure 2 Kaplan-Meier estimate of new active substance 
survival: significant therapeutic advance versus no significant 
therapeutic advance. 



ratio of the latter group of drugs as the benefits they 
offer are significantly lower, whereas the probability that 
they will acquire a serious safety warning is the same. In 
this study, 90.8% (305/336) of drugs were of no add- 
itional significant therapeutic benefit and fell into this 
category. Getting drugs with significant benefits to 
market quickly should be a priority and Health Canada 
should investigate whether its ability to determine what 
type of review is most appropriate for an NAS could be 
improved beyond its current level of 77.4% accuracy. 
Being better able to determine the eventual therapeutic 
benefit could mean that more than 71% (22/31) of 
drugs with significant therapeutic benefits will receive a 
priority review while at the same time having fewer than 
22% (67/305) without significant therapeutic benefits 
getting the same type of resource intensive review. 

Knowing that biologies do not have any greater prob- 
ability of receiving a serious safety warning compared 
with traditional medications is also reassuring as biologies 
now constitute about 25% of all new drugs approved, 15 
and it is quite likely that drug research and development 
will be increasingly turning to biologies. This study 
showed that 27.3% of biologies eventually receive a 
serious safety warning or have to be withdrawn from the 
market. This figure is virtually the same as the 29% 
Kaplan-Meier estimate for a first safety-related regulatory 
action for biologies approved in the USA and the 
European Union between January 1995 and June 2008. 15 

It needs to be noted that the presence or absence of 
regulatory safety warnings is not equivalent to the overall 
safety of a product. An evaluation of overall safety would 
also include an examination of risks detected prior to 
approval, contraindications and warnings about use of a 
drug. However, an examination of the time to the first 
postmarket regulatory warning, the methodology that 
was used in this paper, is consistent with what other 
authors have carried out in analysing the postmarket 
safety profile of drugs in general, 16 specific classes of 
drugs 15 and in comparing different groups of drugs. 14 

This study has several limitations. One possible criti- 
cism is that there might be a systematic difference in the 
frequency of ADR reporting depending on the class of 
the drug so that, to take one scenario, ADRs might be 
under-reported for biologies compared to traditional 
drugs. However, postmarket regulatory action encom- 
passes more than just the receipt and analysis of ADR 
reports. While safety reports are sometimes triggered by 
ADRs, Health Canada also utilises other sources of infor- 
mation in making its decision about issuing a serious 
safety warning. 17 The definition of a serious safety 
warning was based on the way that Health Canada dis- 
played the information (bolded black print and/or 
boxed text), but the criteria that Health Canada used to 
develop its safety warnings and the emphasis that it 
placed on any particular safety issue are not known. 
There were inconsistencies in the Health Canada data- 
bases. Some drugs identified as an NAS in the TPD 
annual reports were not called an NAS in the NOC 



4 



Lexchin J. BMJ Open 201 4;4:e004289. doi:1 0.1 1 36/bmjopen-201 3-004289 



e 



Open Access 



Online Query website. Other drugs listed in the annual 
reports could not be found on the DPD website. On the 
other hand, the information that gatifloxacin was no 
longer marketed in Canada was only found on the DPD 
website. The date on which an NAS receives an NOC is 
not necessarily the date on which the company actually 
decides to market the drug, and therefore the length of 
time the drug is available before it receives a safety 
warning may be shorter than what is reported here. The 
therapeutic value of 70 of the NAS could not be deter- 
mined from the two sources consulted. It is not possible 
to determine whether there were differences in the 
number of people who were potentially harmed by the 
safety problems that triggered the safety warnings for the 
various drugs. Similarly, all safety warnings were treated 
as equivalent regardless of the possible number of 
people affected or potentially affected or the nature of 
the safety issue, for example, a catastrophic side effect 
(efalizumab, progressive multifocal leukoencephalopa- 
thy) or a significant contraindication (dolasetron — any 
therapeutic use under 18 years of age, any postoperative 
nausea; see online supplementary appendix). Once 
again though, this approach is consistent with that used 
by Arnardottir et al, 14 Giezen et al 15 and Lasser et al. 16 
Finally, it is important to note that the regulatory deci- 
sion to issue a safety warning should not be equated 
with the actual degree of harm caused by the drug. 

Although this study relied primarily on Canadian data, 
its conclusions regarding the postmarket safety profile of 
the four groups of drugs examined are likely to be gen- 
eralisable to other countries and regions (eg, Australia, 
the European Union and the USA) with similar drug 
regulatory agencies. The distinction between a drug 
derived from traditional chemical synthesis and a bio- 
logical product is independent of the regulatory jurisdic- 
tion. The method used here to determine the 
therapeutic value of the products relied on objective eva- 
luations from two groups that did not have any conflicts 
of interest. 

One final question that this study raises is whether the 
current level of premarket safety evaluation undertaken by 
Health Canada is acceptable. Depending on the group of 
drugs being examined, between 27.3% and 40.2% eventu- 
ally received a serious safety warning or were withdrawn. 
This is a question that can only be answered through a 
detailed examination of the way that Health Canada 
reviews the clinical trial information that it receives from 
the pharmaceutical companies. At present, Health 
Canada's treatment of this information as commercially 
confidential precludes such an examination. 18 
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